Description and Acquirement of Macro-Actions in Reinforcement Learning
نویسندگان
چکیده
Reinforcement learning is a framing of enabling agents to learn from interaction with environments. It has focused generally on Markov decision process (MDP) domains, but a domain may be non-Markovian in the real world. In this paper, we develop a new description of macro-actions for non-Markov decision process (NMDP) domains in reinforcement learning. A macro-action is an action control structure which provides an agent with control which applies a collection of related microscopic actions as a single action unit. Also we propose a method for dynamically acquiring macro-actions from the experiences of agents during reinforcement learning process.
منابع مشابه
Macro - Actions in Reinforcement Learning : An EmpiricalAnalysisAmy McGovern and Richard
Several researchers have proposed reinforcement learning methods that obtain advantages in learning by using temporally extended actions, or macro-actions, but none has carefully analyzed what these advantages are. In this paper, we separate and analyze two advantages of using macro-actions in reinforcement learning: the eeect on exploratory behavior, independent of learning, and the eeect on t...
متن کاملMacro Actions in Reinforcement Learning An Empirical Analysis
Several researchers have proposed reinforcement learning methods that obtain ad vantages in learning by using temporally extended actions or macro actions but none has carefully analyzed what these advantages are In this paper we separate and an alyze two advantages of using macro actions in reinforcement learning the e ect on exploratory behavior independent of learning and the e ect on the sp...
متن کاملA Method for Learning Macro-Actions for Virtual Characters Using Programming by Demonstration and Reinforcement Learning
The decision-making by agents in games is commonly based on reinforcement learning. To improve the quality of agents, it is necessary to solve the problems of the time and state space that are required for learning. Such problems can be solved by Macro-Actions, which are defined and executed by a sequence of primitive actions. In this line of research, the learning time is reduced by cutting do...
متن کاملA Pilot Study on the Evolution of Reward Signals for Hierarchical Reinforcement Learning
Recent research has shown that reinforcement learning agents can by greatly advantaged of the possibility of learning to select macro actions instead, or beside, fine primitive actions. The route usually followed to exploit this idea is to build agents with hierarchical architectures that can learn both a repertoire of macro actions and a macro policy that selects them, on the basis of the “fin...
متن کاملPlanning with Closed-Loop Macro Actions
Planning and learning at multiple levels of tempo ral abstraction is a key problem for arti cial intelli gence In this paper we summarize an approach to this problem based on the mathematical framework of Markov decision processes and reinforcement learn ing Conventional model based reinforcement learning uses primitive actions that last one time step and that can be modeled independently of th...
متن کامل